SOFA: An extensible logical optimizer for UDF-heavy data flows
نویسندگان
چکیده
منابع مشابه
SOFA: An Extensible Logical Optimizer for UDF-heavy Dataflows
Recent years have seen an increased interest in large-scale analytical dataflows on non-relational data. These dataflows are compiled into execution graphs scheduled on large compute clusters. In many novel application areas the predominant building blocks of such dataflows are user-defined predicates or functions (Udfs). However, the heavy use of Udfs is not well taken into account for dataflo...
متن کاملMAO - An extensible micro-architectural optimizer
Performance matters, and so does repeatability and predictability. Today’s processors’ micro-architectures have become so complex as to now contain many undocumented, not understood, and even puzzling performance cliffs. Small changes in the instruction stream, such as the insertion of a single NOP instruction, can lead to significant performance deltas, with the effect of exposing compiler and...
متن کاملControl of an Extensible Query Optimizer: A Planning-Based Approach
III this paper we address the problem of controlling the execution of a query optimizer. We describe a control for the optimization process that is based on planning. The controller described here is a goal-directed planner that intermingles planning with the execution of query transformations, and uses execution results to direct further planning of optimizer processing. We describe this contr...
متن کاملUser-Extensible Simplification - Type-Based Optimizer Generators
For abstract data types (ADTs) there are many potential optimizations of code that current compilers are unable to perform. These optimizations either depend on the functional specification of the computational task performed through an ADT or on the semantics of the objects defined. In either case the abstract properties on which optimizations would have to be based cannot be automatically inf...
متن کاملAn Architecture for Compiling UDF-centric Workflows
Data analytics has recently grown to include increasingly sophisticated techniques, such as machine learning and advanced statistics. Users frequently express these complex analytics tasks as workflows of user-defined functions (UDFs) that specify each algorithmic step. However, given typical hardware configurations and dataset sizes, the core challenge of complex analytics is no longer sheer d...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Information Systems
سال: 2015
ISSN: 0306-4379
DOI: 10.1016/j.is.2015.04.002